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Abstract 

An inherent characteristic of technology education is the continual develop- 
ment of new technologies and creating innovative applications of already ex- 
isting technologies. As exciting as these innovations can be, technology edu- 
cators and school staffs are frequently challenged to accomplish high levels 
of implementation. The metaphor of the Implementation Bridge and four 
research-based constructs of the Concerns Based Adoption Model are intro- 
duced in this article. Each can be used to evaluate the extent of implementa- 
tion and as diagnostic tools for facilitating implementation. Each also can 
be applied in studies of relationships between extent of implementation and 
student outcomes. (Keywords: change, implementation, evaluation, leaders, 
Concerns-Based Adoption Model, CBAM) 


The hardest thing about technology is not selling them on it. 

It is getting them to use it! 

— Phil H., district tech coordinator (reflecting on the importance of understanding 
that change is a process) 

T his quote summarizes much that is currently understood about the 
significance of technology as an innovation and the challenges en- 
tailed in the widespread presence of new forms and integrated uses of 
technology in today’s schools and classrooms. Cubans (1986) thoughtful re- 
view of the history of the introduction of various technology innovations in 
education is an important reminder that developing a new form or process 
does not guarantee that it will be widely used. The continuing challenge with 
technology innovations is to move beyond their early adoption by technol- 
ogy enthusiasts and scale up to widespread use. 

An exciting array of technology resources is available to today’s teachers 
and classrooms. Most classrooms have access to laptops for students and 
teachers, document projectors, digital photos and video, interactive white- 
boards, sound systems, and the Web. However, how each is used and the 
extent of use by teachers and students varies dramatically. For example, in 
many classrooms, use of the interactive whiteboard is limited to projection 
of teacher-generated information, and PCs are used only for remediation 
or free time. In other classrooms, there is true integration of the available 
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technologies and extended applications. The whiteboard is regularly con- 
nected to the Web. Students create their own video productions and develop 
digital portfolios. 

There is also a gap between students’ use of technology outside the class- 
room (e.g., texting, MP3 players, and social networking) and what they do in 
the classroom (e.g., printing documents). Many promising technologies are 
widely available, but the Achilles heel is a lack of understanding about what 
is involved in helping teachers to fully implement and integrate their uses. 

This challenge takes place at the nexus of technology innovation develop- 
ment and the dynamics of personal and organizational change. Development 
of a promising technology does not guarantee that it will achieve widespread 
use. Teachers will vary in their interest in adopting a new approach and in 
their competence to use it. This is where constructs and tools from change 
process research can be instructive. The extent and quality of use of new 
approaches can be greatly enhanced when there is understanding of how 
people change. Regardless of the potential power of a technology, until it is 
used and used well, the promised outcomes will not be attained. 

This paper begins with the assumption that various technology innova- 
tions have been developed and that there is interest in achieving widespread 
and appropriate use. The constructs introduced herein also can be used to 
evaluate impact and determine the necessary actions for going to scale. The 
article will also introduce research-verified constructs and measures, along 
with related findings, to illustrate how the dual challenges of evaluating an 
innovations impact on outcomes and achieving its widespread use. The 
concluding discussion draws implications for research, evaluation, and 
facilitation of implementation. 

Four Simple Questions 

Regardless of the potential promise of any technology innovation, special- 
ized effort is needed to assure widespread and effective use. An important 
beginning point is accepting as fact that different implementers are not likely 
to use the technology exactly as the developer envisioned. Exact replication 
from classroom to classroom is highly unlikely. Therefore, as an innovation 
is disseminated, each prospective user should ask four simple questions: 

1. Is it being used? 

2. How well is it being used? 

3. What factors are affecting its use/nonuse? 

4. What are the outcomes? 

As obvious as each of these questions may seem, they really are too 
simplistic. For example, one of the well-established perspectives for under- 
standing change is Diffusion of Innovation Model (Rogers, 2003). In this 
perspective everyone does not “adopt” the innovation at the same time. 

Some are quick to try the new way, whereas others deliberate and delay. In 
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one set of classic diffusion studies, researchers examined characteristics of 
farmers and their decision to purchase hybrid corn seed instead of planting 
seed they had stored from last year’s crop (Ryan & Gross, 1943). Some were 
quick to try the new way, but others waited to see how the new way worked 
out. The researchers identified a set of characteristics that distinguished 
those who were first to adopt a new approach from those who took longer. 
Five adopter categories were identified ranging from Innovators, to Early 
Adopters, to Laggards. To have all adopter categories in this study make the 
change — in other words, to achieve widespread use of hybrid corn seed — 
took years. 

The decision to use a particular technology innovation in schools is more 
complicated. First of all, the decision to adopt most technology innovations 
is made by administrators, not teachers. Second, becoming skilled in using 
a new form of technology takes time. In addition, whereas planting hybrid 
corn seed requires no new machinery or procedures, most new applications 
of technology are more complex and may require new infrastructure, such 
as greater memory or bandwidth. Also, most of the time teachers will benefit 
from training in how to use the new technology. 

Another important difference between agricultural settings and schools 
has to do with what it means to be a user. In agriculture, where the Diffusion 
of Innovation Model bloomed, farmers either “adopted” hybrid corn seed 
or planted the seed they saved from last year. Determining who was using 
the innovation was straightforward. Adoption of the innovation was di- 
chotomous; it was used or it was not used. In contrast, teachers and schools 
becoming high-quality users of technology innovations is a process, not an 
event. In other words, teachers and schools are not non-users of a particular 
technology one day and expert users the next day. There is a gradual pro- 
cess of trial and error as each implementer learns how to use the new tool, 
process, or function. 

Unfortunately, too many research and evaluation efforts implicitly assume 
the agricultural type of dichotomy of use/non-use. For example, the Gold 
Standard research design requires random assignment of subjects to treat- 
ment and control groups. Most program evaluation studies assign teachers 
or schools to program and comparison groups. The assumption in the as- 
signments is that one group uses the innovation and the other does not. This 
is a misleading view of how change works on the ground. It is much more 
complicated for teachers and schools to accomplish high levels of implemen- 
tation, and rarely are the comparison/control groups pure. An implication 
of this phenomenon is that conclusions about the effects of the innovation 
are suspect when researchers/ evaluators haven’t defined and verified that 
implementation occurred in the experimental group and is not present in 
the control group. 

Another of the change perspectives, the Goncerns Based Adoption 
Model (GBAM) (Hall & Hord, 2011), addresses the messiness of real-world 
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Figure 1. The Implementation Bridge. 


implementation. The CBAM offers several research-based constructs and 
tools that can be used to understand, facilitate, and evaluate the more com- 
plex efforts entailed with introducing technology innovations in classrooms 
and schools. An important beginning point with the CBAM perspective 
is the assumption that change is a process, not an event (Hall, Wallace & 
Dossett, 1973; Hall & Hord, in press). Rafher than assuming use is dichoto- 
mous (use/nonuse), in this perspective becoming a competent and confident 
innovation user is a developmental phenomenon that takes time. Four of the 
CBAM constructs that address this complexity will be introduced below, but 
first a metaphor. 


The Implementation Bridge 

As emphasized above, an important consideration about changes in educa- 
tion is appreciating the complexity, sophistication, and subtlety of most of 
today’s innovations. For most innovations, change should not be considered 
in terms of adoptions. Instead change needs to be thought about as a process 
of implementation. For example, technology experts may easily pick up 
and use a new device or application, but for the typical teacher or school, 
implementing the new way may not be easy. Integrating the uses of several 
technologies and applications in the milieu of the classroom is even more 
complicated and challenging. 

There is often insufficient appreciation of how complex the implemen- 
tation processes can be. This is the metaphorical equivalent of asking 
implementers to back up, take a running start, and attempt to leap across 
the chasm from past practice to full use of a new way. That chasm can be 
deep, and the distance from one side to the other can be very long. To help 
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illustrate these realities of a change process, in 1999 the author introduced 
the metaphor of an “implementation bridge” (see Figure 1). The challenge 
of accomplishing sustained and widespread integrated uses of technology 
is the metaphorical equivalent of providing a bridge to facilitate teachers 
and schools progressing across a bridge. Instead of expecting teachers and 
schools to make giant leaps across the chasm, providing an implementation 
bridge will result in more frequent and higher- quality use. 

The implementation bridge metaphor addresses another important 
component: outcomes. The explicit assumption with most innovations is 
that if they are used properly, there will be higher outcomes (i.e.., increases 
in student learning). When the perspective of an implementation bridge is 
employed, outcomes can be expected to vary with how far across the bridge 
each implementer has progressed. In theory, those that are farther across the 
bridge should have higher outcomes. This hypothesis has been found to be 
true in some studies (see, for example, George, Hall, & Uchiyama, 2000). 

Four Refined Questions 

When change is accepted as a process requiring the equivalent of an imple- 
mentation bridge, and technology innovations are considered complex, 
the four simple questions introduced above can be refined and made more 
nuanced; 

1. How can the change process be facilitated to achieve high levels of imple- 
mentation in classrooms and across a school? 

2. What factors and approaches can be applied for achieving widespread use? 

3. What is the extent of implementation with each individual and school? 

4. How do outcomes vary with extent of implementation? 

The remainder of this paper will introduce constructs, measures, and 
research findings from studies of implementation and use them to describe 
and illustrate ways to address each of these questions. 

Three Diagnostic Dimensions for Assessing and Faciiitating Impiementation 

The three rows on top of the implementation bridge in Figure 1 represent 
the three Diagnostic Dimensions of the Concerns Based Adoption Model 
(Hall & Hord, 2011). Each has an extensive history of research and evalu- 
ation applications. Each construct can also be used by researchers, evalua- 
tors, leaders, and implementers. This means that the same construct and its 
data can do double duty: It can be used to document the current extent of 
implementation and as diagnostic information for planning interventions 
to further facilitate implementation. In other words, the researcher/ evalua- 
tor can use information from each dimension to measure how far across the 
bridge each implementer has progressed. Change facilitators can also use the 
same constructs and information for planning and making interventions to 
help implementers move further across the bridge. 
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Table 1 . Levels of Use (LoU) of the Innovation: Indicators 


VI Renewal 
V Integration 
IVB Refinement 
iVA Routine 
iii Mechanicai Use 

ii Preparation 
i Orientation 
0 Nonuse 


Use 

The user is seeking more effective alternatives to the established use of fhe innovafion. 

The user is making deliberate efforts to coordinate with others in using the innovation. 

The user is making changes to increase outcomes. 

The user is making few or no ehanges and has an established pattern of use. 

The user is using the innovation in a poorly coordinated manner and is making user-oriented changes. 
Nonuse 

The person is preparing to use the innovation for the first time. 

The person is seeking out information about the innovation. 

No action is being taken with respect to the innovation. 


Hall &Hord (20ft) 


Use and Nonuse: Using an Innovation is not Dichotomous 
Leveis of Use 

As noted above, learning to use technology innovations, as well as most 
other education and business innovations, is not as simple as is assumed in 
the various “yes/no” change-is-an-event perspectives. Rather than being di- 
chotomous (use or nonuse), change process researchers have established that 
there are different Levels of Use (see Table 1). Levels of Use (LoU) describe 
eight operationally defined behavioral profiles. There are three ways of being 
a nonuser of an innovation and five behavioral profiles of users. The estab- 
lished method for measuring Levels of Use is a focused interview protocol 
that has been research verified and applied in many studies (Hall & Loucks, 
1977; Hall, Dirksen, & George, 2006). 

The Levels of Use can be used to describe the current state of each 
implementer. The LoU assessment information can be used diagnostically in 
making interventions that help the teacher(s) move to a more advanced level 
of use. For example, the person at Nonuse (LoU 0) is not doing anything in 
relation to the innovation. Even if they are expected to be using it, they are 
taking no innovation-related actions. Interventions aimed at this LoU would 
be intended to move them toward engagement with learning about use 
(LoU I: Orientation), whereas persons at Orientation (LoU I) are looking 
for information about the innovation. They have not decided to use it, which 
would place them at Preparation (LoU II). They may be talking with others 
or searching for information online, but they still are nonusers. This is a time 
to provide descriptive information about the innovation and encouragement 
to consider its use. 

Most first-time users of an innovation will be at Mechanical Use (LoU 
III). Their behaviors are related to organizing/scheduling and attempting 
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to make everything work smoothly. The primary focus is on just making 
it work. Teachers at Mechanical Use with implementing technology are 
struggling with management acts such as distributing devices, continually 
referring to the user manuals to see which icon on the toolbar will do what 
they want to do, and grouping students to maximize efficiencies. Even tech 
experts can return to mechanical use when the new technology/ application 
is less familiar or poorly documented. Although it might be expected that in 
the second year of use most would move beyond Mechanical Use, this is not 
what longitudinal studies have found. Many teachers in the second and even 
third year of implementation will still be at Mechanical Use. 

To achieve continued use of an innovation, it is best for teachers to 
move beyond Mechanical Use. The Routine Use (LoU IVA) level users have 
repeatable procedures in place and can predict where they will be next week 
and beyond. Some will move to one of the higher Levels of Use by mak- 
ing adaptations in their use with the intention of increasing outcomes. The 
typical time period for teachers in a school to move from Nonuse to Routine 
Use and higher is three to five years, if not longer (see for examples: Hall & 
Loucks, 1977; Hall & Hord, 2011; Thornton, E., & West, 1999). Of course, 
the amount of time it takes dependa on a number of factors, such as the size 
of the innovation, the amount of implementation support, and what other 
changes are being implemented at the same time. 

Levels of Use is one way to describe and measure the extent to which each 
implementer has moved across the implementation bridge. With technol- 
ogy innovations, each of these levels is easily imagined. For example, some 
teachers do not look at what is new or could be done differently (LoU 0: 
Nonuse). Other teachers explore what a new application could do, but they 
do not decide to use it at this time (LoU I: Orientation). LoU III Mechanical 
users consume a great deal of tech support attention and time. These are the 
implementers who call the technology coordinator daily seeking help with 
finding certain subroutines because they cant find the right icon or forget 
that you need to right click for certain functions (LoU III: Mechanical Use). 
Of course, the ideal users are the two teachers at LoU V: Integration. They 
are the ones who have been using interactive whiteboards individually and 
now decide to work together to integrate their use and combine their stu- 
dents to achieve higher student outcomes (LoU V: Integration). 

Assessing Extent of Implementation with LoU 

Another application of Levels of Use is in evaluation and research stud- 
ies. Levels of Use can be used to document how far across the bridge each 
implementer has progressed. This information about extent of implementa- 
tion can be related to student outcomes and other factors such as the change 
leadership style of the principal. 

An important methodological implication of LoU has to do with the 
assignment of teachers/schools to so-named treatment and control groups. 

Volume 42 Number 3 | Journal of Research on Technology in Education | 237 
Copyright © 201 0, ISTE (International Society for Technology in Eduoation), 800.336.51 91 
(U.S. & Canada) or 541 .302.3777 (Int'l), iste@iste.org, www.iste.org. All rights reserved. 



Hall 


The long-established practice of random assignments does not make the 
two groups pure. The researcher/ evaluator needs to carefully check at the 
individual level in both groups to be certain that there are no nonusers 
in the treatment group and no users in the control group. Otherwise, it is 
very likely that both groups will in fact be composed of a mix of users and 
nonusers. Without documentation of the purity of each study group, doubts 
should be raised about conclusions that may be drawn about the effects of 
the innovation. For example, in one school district program evaluation with 
11 project schools and 11 comparison schools, the evaluators concluded 
that there were no significant differences in student test scores. When LoU 
was measured, it turned out that both groups were made up of users and 
nonusers. Only 80% of the teachers in the project schools were using the in- 
novation, whereas 49% of the teachers in the comparison schools were users 
(Hall & Loucks, 1977). No wonder the school district evaluators concluded 
that there were no significant differences between the two schools. 

Levels of Use and Student Outcomes 

Addressing the fourth refined question, examining relationships between 
extent of implementation and student learning, is more work. The Level 
of Use of each implementer has to be measured and then related to one or 
more measures of student outcomes. To have representation of most levels 
usually requires longitudinal study designs and several years of effort. An 
informative example of this type of a study is one that examined constructiv- 
ist teaching of mathematics. This innovation represented a paradigm change 
from teacher-led instruction to teacher- as -facilitator of student construction 
of understanding. For example, instead of accepting a single teacher-taught 
way of calculating, teachers were to encourage students to use different ap- 
proaches. In one school district, LoU was assessed each of the first two years 
of implementation. This particular district -wide implementation project had 
strong leadership from the superintendent and three master teachers as- 
signed to facilitate implementation (Johnson, 2000). Level of Use of over 100 
teachers was measured each year. 

In this study, the relationships between student outcomes and how far 
across the bridge teachers had progressed was nearly linear. The highest 
outcomes were found with two teachers at LoU V Integration. This LoU is 
where two or more implementers are working together to make changes in 
their efforts to increase student outcomes. The lowest level of student out- 
comes was associated with teachers at Mechanical Use (LoU III). Teachers 
at Routine Use (LoU IVA) had slightly higher outcomes than the teachers at 
Refinement (LoU IVB) (George, Hall, & Uchiyama, 2000). 

A Further Implication of LoU for Facilitating Implementation 

The first of the four refined questions in this paper asks what can be done to 
facilitate further implementation. Leaders of change efforts can use Levels 
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of Use as a diagnostic tool. The same information that is collected to address 
research/ evaluation questions can inform change facilitation. For example, 
in the above-referenced mathematics study, 20-t% of the teachers remained 
at Mechanical Use (LoU III) at the end of two years. This information as 
a diagnostic would suggest that these teachers need individually targeted, 
purposeful interventions to move them to higher Levels of Use. 

Levels of Use are easy to envision with technology innovations. For 
example. Mechanical Use is all too frequently observed and talked about. 
This Level of Use can be observed, and experienced, with relatively simple 
technology changes, such as upgrades in e-mail as well as when complex 
system changes are introduced, as with an all new data management system. 
Depending on the complexity of the change and the amount of implementa- 
tion facilitation support, the time period for Mechanical Use (the length of 
the implementation bridge) will be shorter or longer. 

Fidelity of Implementation: More Important than Ever 
Innovation Configurations 

Another critical implementation question to address is how close the practic- 
es of each implementer match with the vision of the innovations architect. A 
colleague, Ed Caffarella, tells the story of what happened on his campus when 
the school was first introducing PCs. The campus provost distributed PCs to 
10 faculty members and encouraged them to explore ways to use them. Most 
did the expected things, such as word processing and computations. How- 
ever, a faculty member in the art department took his PC apart and made a 
mobile! All 10 PCs were being used, but there was tremendous variation in 
how closely each application fit with what the manufacturer had envisioned. 

In change process research, the construct that is used to address these 
variations in how an innovation is made operational is called Innovation 
Configurations (IC). Different configurations of an innovation can exist 
in different classrooms within a school and most certainly from school to 
school. When asked, each implementer is likely to say, “Oh yes, I am using 
XYZ technology.” But what they are doing and which components of the 
innovation are being used can range from exact replications of what the de- 
veloper had in mind to a practice that is unimaginable to the developer. This 
means that in practice there will likely be a very wide range of configurations 
under the umbrella of the innovation’s name. 

For example, in the K-12 environment, Donovan (2007) found that with- 
in one middle school in which a 1:1 laptop program was being implemented, 
there were three different implementation “models” or configurations. 
Interestingly, some teachers represented more than one model, highlight- 
ing the phenomenon that implementation of technology in schools may not 
only involve more than one configuration, but some implementers may use 
different configurations with different students. 
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Innovation Configuration Map for the 
Mathematics Program 


3) Teacher Poses Mathematical Tasks/lnvestigations {poses, frequency, 
openOended questions, language} 


Teacher poses open-ended 
problem, highlights 
mathematical aspects and 
asks students to determine 
how to figure them out. 
Open-ended questions are 
used to pose problems, not 
only at the beginning but 
also throughout the lesson. 
Teacher uses mathematical 
language to present 
tasks/investigations. 


b 

Teacher identifies 
mathematical aspects of 
tasks/investigations and 
explains how to figure them 
out. Teacher directions are 
clear. Some mathematical 
language is used. Some 
open-ended questions are 
asked. 


c 

The teacher presents the 
activity with little or no 
explanation. Teacher uses 
little or no mathematical 
language. Some teacher 
directions are clear. Nearly 
all questions require one- 
word answers. 


d 

Teacher structures activity 
and directs students/ 
activity. Questions requiring 
one-word answers are 
used to check for student 
understanding. Isolated use 
of math vocabulary. 


Teacher presents/explains 
isolated concept or 
procedure and assigns 
individual student work. 
Questions requiring one- 
word answers are used to 
check for student 
understanding. Isolated use 
of math vocabulary. 


Figure 2. Example Innovation Configuration Map Components. Component from 1C map for Teaching and Learning Mathemat- 
ics (Alguist & Hendrickson, 1 999, p. 22). 


In this time of high emphasis on use of “evidence-based” practices, fidel- 
ity considerations are more important than ever. If a technology has been 
verified to produce certain outcomes, then future implementers are going to 
expect to obtain the same outcomes. This requires two sets of information: 


1. The developers must identify and provide information about what a 
high-fidelity implementation configuration of their innovation looks like. 

2. Implementers must strive to put in place the high-fidelity 
configuration(s). 

Innovation Configuration Mapping 

The construct of IC is easy to imagine. Answering both questions requires a 
methodology for measuring fidelity of implementation. A generic method 
for assessing configurations has been developed. The approach uses the 
metaphor of a map. A map provides the information for getting from point 
A to point B. One may take an interstate highway, state highways, back roads, 
or maybe a train or a walking trail. An Innovation Configuration map is 
designed to describe the different ways that the various components of an in- 
novation could be implemented (Hord, Stiegelbauer, Hall, & George, 2006). 

Developing an IC map requires intensive study of all of the materials and 
resources associated with the innovation, interviewing the developer, and 
observing a range of sites. The field work needs to cover the range from ideal 
implementations to efforts that the developer judges to not be representative 
of the innovation. 

All of the information is first used to identify key components of the 
innovation. To provide a simple example, consider student grouping to be 
a component. The next step is to identify different possible variations of 
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Student control of laptops during learning experiences (control, position, decision making) 

c 


a 

Laptops are centrally posi- 
tioned and students share 
control in relatively equal 
proportions during learning 
experience. It is difficult to 
determine whose laptop it 
is. Students consult with 
each other on navigation 
and aesthetics of content. 


b 

Laptops are centrally 
positioned, yet it is apparent 
that one student is sharing 
with the others. Control of 
laptop is generally by one 
person, but consultation on 
navigation and aesthetics of 
content is evident. 


Laptops are positioned in 
front of one individual but 
turned tor others to see. 
Control is by one person, 
and others are observers. 


e 

No sharing of laptops. 
Students who do not have 
laptops use alternative 
learning tools. 


Figure 3. 1C map Component for One-to-One Laptop Program in the Middle School (Donovan & Green, 2009, n.p.). 


the component. For example, students could work independently, in pairs, 
in small groups, or as a whole class. On an IC map, each variation is to be a 
“word picture” description of one way that component could be implemented. 

The variations of a component are displayed as in a rubric, but with 
several important differences. Unlike in most rubrics, the ideal variation is 
labeled “a” and placed to the left-hand side. The remaining variations are 
laid out from left to right to reflect decreasing fidelity of that component and 
labeled accordingly: “b,” “c,” “d,” etc. Another key difference from traditional 
rubrics is that the variations to the right-hand side don’t just describe less of 
“a.” These variations describe what has been implemented instead of what is 
described in the “a” variation. Figure 2 is an example component from the 
earlier referenced mathematics study. Figure 3 is a sample component from 
Donovan’s IC Map of Student Uses of Laptops. 

Innovation Configuration Findings 

Once fully developed, an IC map can be used as a tool for assessing degree of 
implementation and as a diagnostic for planning further implementation of 
supporting interventions. In the typical study, collecting the data to complete 
an IC map requires a combination of classroom observations and teacher 
interviews. The IC map becomes the summary document for what has been 
observed and heard about regarding how the innovation is implemented. 

The circled variations for each component on an IC map will represent that 
teacher’s configuration. The IC map information also can be summarized for 
all teachers by component to provide an overall picture of how far across the 
bridge implementation has progressed. For example, six teachers were using 
the “a” variation of Component 6, five were using the “c” variation, and two 
were using the “f” variation. Also, cluster analysis techniques can be applied 
for identifying which combination of components and variations are most 
typically implemented (Hord, Stiegelbauer, Hall, & George, 2006). 

Donovan (2007) found multiple and vastly different configurations of 
laptop program implementation in one middle school. In one configuration, 
students were using laptops for authentic project-based learning, whereas in 
another, students were using the laptops for more limited purposes — word 
processing and as a calculator. This second configuration could lead people 
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to believe the laptops were not a valuable investment. They certainly would 
appear to contribute to different student outcomes. It would be unfair to 
draw summative conclusions about student use of laptops without docu- 
menting the configurations implemented in each classroom. 

Exploring relationships between Innovation Configurations and student 
outcomes can be done. Planning for this analysis is one of the structural rea- 
sons for having the ideal variation named “a.” The number of variations may 
be different for different components, but if “a” always represents the ideal, 
then data analyses are easy to follow. With cluster analyses, it is possible to 
distribute the configurations from high to low fidelity. Those classrooms 
with more “a” variations represent higher fidelity, whereas those with more 
“e” and “f” variations represent low fidelity. 

Keeping with the mathematics study example, it was found that those 
classrooms with higher-fidelity configurations were associated with higher 
student outcomes. This will not always be the case, but in this study there 
were significant advantages in terms of student outcomes when the innova- 
tion was used with fidelity (George, Hall, & Uchiyama, 2000). 

In the case of the laptop program at the middle school, exploring rela- 
tionships with outcomes was not an integral part of the study design, but 
during IC map development it became evident that a component of the IC 
map was student off-task behavior during technology-based lessons. When 
the relationship between student off-task behavior and individual configu- 
rations was explored more closely, Donovan (2007) was able to determine 
that increased access did not necessarily equate to increased technology- 
based off-task behavior. Of the three configurations, the “b” configuration, 
in which the students used the laptops relatively consistently, but not always 
for authentic learning tasks, had more frequency of student off-task behavior 
and a greater variety of off-task behaviors (e.g., Web surfing, e-mail, games) 
than the “c” configuration, in which students sporadically used the laptops, 
and the “a” configuration, in which students used the laptops for the majori- 
ty of their class time. Oddly enough, this highlights the relationship between 
fidelity of technology implementation and desired outcomes, but only when 
we look at all three configurations. 

Implications of IC for Facilitating Impiementation 

One of the important uses of IC maps is to make them available to the 
implementers. All too often, teachers are provided with abstract descriptions 
of what the innovation should look like when it is used with fidelity. De- 
velopers are much more effective at describing the innovations philosophy, 
outcomes, and implementation requirements than what it looks like on the 
ground. An IC map presents clear descriptions, component by component, 
of what use of the innovation can look like. An IC map also is useful to the 
change facilitators, coaches, and principals, who are supposed to know what 
to look for when they are observing the innovation in use. 
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Table 2. Stages of Concern (SoC) about the Innovation (George, Hall, & Stiegelbauer, 2006; Hall & Hord, 201 1 ). 


6 Refocusing 

5 Collaboration 
4 Consequence 
3 Management 

2 Personal 


1 Informational 


0 Unconcerned 


The focus is on the exploration of more universal benefits from the innovation, including the possibility 
of major changes or replacement with a more powerful alternative. Individual has definite ideas about 
alternatives to the proposed or existing form of the innovation. 

The focus is on coordination and cooperation with others regarding use of the innovation. 

Attention focuses on impact of the innovation on students in his/her immediate sphere of influence. 

Attention is focused on the processes and tasks of using the innovation and the best use of informa- 
tion and resources. Issues related to efficiency, organizing, managing, scheduling, and fime demands 
are utmost. 

Individual is uncertain about the demands of the innovation, his/her inadequacy to meet those 
demands, and his/her role with the innovation. This includes analysis of his/her role in relation to the 
reward structure of the organization, decision making, and consideration of potential confliets with 
existing structures or personal commitment. Finaneial or status implications of the program for self 
and colleagues may also be reflected. 

A general awareness of the innovation and interest in learning more detail about it is indicated. The 
person seems to be unworried about himself/herself in relation to the innovation. She/he is interested 
in substantive aspeets of the innovation, sueh as general characteristics, effects, and requirements for 
use, in a selfless manner. 

Little concern about or involvement with the innovation is indieated. 


The Personal Side of Change: Feelings and Perceptions Count 

The third diagnostic dimension for understanding implementation ad- 
dresses the affective part. The people who are engaged with getting across 
the implementation bridge experience feelings, perceptions, frustrations, 
and moments of joy. The emotional part of change often is neglected, with 
resulting arousal of unnecessary resistance to the innovation. 

Stages of Concern 

The construct for describing the affective part of change is called Stages of 
Concern about the Innovation. There is a long history of research and ap- 
plication of this construct and its measures. Many of the studies have been 
done with technology innovations. Four basic areas of concern have been 
identified: 

1. Impact. The focus is on how fhe innovation is affecting students and what 
can be done to increase outcomes. 

2. Task. Time, logistics, schedules, and fitting everything in that must be 
done are of concern. 

3. Self. Personal feelings of uncertainty, whether one can succeed with this 
innovation, and whether the supervisor will support the efforts are cen- 
tral in thought. 

4. Unconcerned. Other things are of more concern at this time than the 
innovation. 
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These four areas of concern have been further divided, as summarized in 
Table 2 (p. 243). There are two parts to Self concerns, Stage 1: Informational 
and Stage 2: Personal. Three distinct parts to Impact concerns have been 
identified. Stage 4: Consequence, Stage 5: Collaboration, and Stage 6: Refo- 
cusing. At any time, each implementer will have some combination of these 
concerns, with some being more intense and others not being intense at all. 

Stages of Concern Findings 

There is a long history of study around Stages of Concern (SoC) beginning 
with the pioneering studies of teacher education students by Frances Fuller 
(1969). One theme across the studies is that, if there is appropriate change 
support and time (three to five years), there will be progression across the 
different SoC. In other words, as implementers move across the implementa- 
tion bridge, their concerns will progress from Unconcerned, to Self, to Task, 
and ultimately to Impact. However, there is no guarantee that this will hap- 
pen (Hall & Hord, in press). 

For example, Donovan and Green (2009) found that even with offers for 
support, some of the teacher educators involved as faculty in a 1:1 laptop 
teacher certification program did not shift in their concerns at all, whereas 
others’ concerns shifted from Self to Impact. 

There are three ways to assess concerns. Change facilitators in the field 
can use an informal conversation called a One-Legged Interview. It doesn’t 
take much time and is not threatening. A second way is the Open-Ended 
Concerns Statement (Newlove & Hall, 1976), which asks respondents in 
their own words to write their concerns. For research and evaluation pur- 
poses, the 35-item Stages of Concern Questionnaire (SoCQ) is available. A 
new form of the SoCQ (Form 075) has recently been established that has 
demonstrated reliability and validity. The scoring of the SoCQ provides a 
concerns profile that indicates the relative intensity of concern for each stage 
(George, Hall, & Stiegelbauer, 2006). 

Implications of SoC for Faciiitating Impiementation 

SoC is particularly relevant to the first of the four refined questions listed 
above. Interventions that facilitate implementation need to be concerns 
based. For example, responding to the old refrain of “computer fear” begins 
with understanding that Self concerns are high. Responses need to be more 
supportive of the person (Stage 2: Personal) and provide answers (Stage 1: 
Informational) to reduce the uncertainty. The time to offer how-fo-do-if 
tips is when implementers are out on the bridge and Task concerns are high. 
This is not the time for a lecture on the philosophy of why the new way 
is better. A related intervention study by Dobbs (2004) using SoC docu- 
mented the importance of training approaches for implementing distance 
education. 
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Table 3. Change Facilitator Styles (Hall & Hord, in press). 

Initiators have clear, decisive, long-range policies and goals that transcend but include implementation of the current innovation. 
They tend to have very strong beliefs about what good schools and teaching should be like and work intensely to attain this vision. 
Deeisions are made in relation to their goals for the school and in terms of what they believe to be best for students, which is 
based on current knowledge of classroom practice. Initiators have strong expectations for studenfs, teachers, and themselves. 

They eonvey and monitor these expectations through frequent contacts with teachers and setting clear expectations of how 
the school is to operate and how teachers are to teach. When they feel it is in the best interest of their school, particularly the 
students, initiators will seek changes in district programs or policies, or they will reinterpret them to suit the needs of the school. 
Initiators will be adamant but not unkind. They solicit input from staff and then make decisions in terms of the goals of the school, 
even if some are ruffled by their directness and high expectations. 

Managers place heavy emphasis on organization and control of budgets, resources, and the correct applications of rules, proce- 
dures, and policies. They demonstrate responsive behaviors in addressing situations or people, and they initiate actions in support 
of change efforts. The variations in their behavior are based in the use of resources and procedures to control people and change 
processes. Initially new implementation efforts may be delayed if they see that their staff are already busy and that the innovation 
will require more funds, time, and/or new resources. Once implementation begins. Managers work without fanfare to provide basic 
support to facilitate teachers! use of the innovation. They keep teachers informed about decisions and are sensitive to exeessive 
demands. When they learn that the central office wants something to happen in their school, they become very involved with their 
teachers in making it happen. Yet they do not typically initiate attempts to move beyond the basics of what is required. 

Responders place heavy emphasis on perception checking and listening to people’s concerns. They allow teachers and others 
the opportunity to take the lead with change efforts. They believe their primary role is to maintain a smoothly running school by 
being friendly and personal. They want their staff to be happy, get along with each other, and treat students well. They tend to 
see their school as already doing everything that is expected and not needing major changes. They view their teachers as strong 
professionals who are able to carry out their instructional role with little guidance. Responders emphasize the personal side of their 
relationships with teachers and others. They make deeisions one at a time and based on input from their various discussions with 
individuals. Most are seen as friendly and always having time to talk. 


Another Key Factor: Leadership Makes a Difference 

The above descriptions have introduced briefly three research-based con- 
structs and tools for measuring how far across the implementation bridge 
each implementer and/or group of implementers has progressed. The bridge 
metaphor and the three diagnostic dimensions offer ways to think about, 
study, and facilitate change processes. An important reminder is that, in eval- 
uation and research studies. Levels of Use, Innovation Configurations, and 
Stages of Concern information needs to be determined for each implementer. 
Each individual in any comparison or control group also should be assessed. 
The individual assessments can be aggregated to view how groups are doing. 

As important as each of the dimensions is individually and in combina- 
tion, there is another factor that drives change success. The importance of at- 
tending to the fact that individual implementers are part of an organization 
and that there are organization factors that affect the rate, as well as whether 
or not implementers make it across the bridge, are too often neglected. 
Teachers are part of a school. Factors within the school can significantly af- 
fect implementation success. 

Perhaps the most significant school-level factor affecting teacher imple- 
mentation success is the leadership role the principal plays. Some principals 
provide strong leadership and support for teachers as they engage with 
implementation. Some principals closely attend to providing materials and 
other resources. Some cheerlead and leave the details to their teachers. Each 
approach represents a different style of leadership, and that style is highly 
correlated with teacher implementation success. 
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Three Change Facilitator Styles 

Researchers have identified three different approaches to change leader- 
ship that principals may exhibit. These do not represent every possible style, 
but three that have been found frequently and studied in depth. The three 
Change Facilitator Styles (CFS) are called Initiators, Managers, and Respond- 
ers (see Table 3, p. 245)). Principals of each style have been studied in several 
countries, including Australia (Schiller, 1991), Belgium (Vandenberge, 1988), 
the United States (Hall, Rutherford, Hord, & Hulling, 1984), and Taiwan 
(Shieh, 1996). Although only the Schiller study was done with technology, 
the same basic descriptions and approaches have been found to fit with many 
education innovations and settings. The consistent finding is that principal 
leadership is a critical factor related to implementation success. 

Initiators have a strong set of ideas about what their school should be like 
and what it should become. They have a passion for the school, and they will 
push their teachers to do all that is needed. Managers focus on following 
the rules and controlling resources. They are skilled at working within the 
organization, getting the most out of budgets, and keeping everything well 
organized. Responders focus on listening to the concerns of their staff. They 
do nol feel lhal ihey have lo do everything themselves. They want everyone 
to be happy and to get along (Hall & Hord, in press). 

Findings from Research 

The reason for introducing Change Facilitator Styles (CFS) in this discussion 
is the consistent findings about the relationships between CFS and teacher 
implementation success. The findings are strong and consistent across inno- 
vations that depending on the CFS of the principal, teachers will have differ- 
ent degrees of success in implementing new programs. The US. studies have 
focused on curriculum innovations, the Australian study examined imple- 
mentation of technology, the Belgian study was of restructuring primary 
schools, and the Taiwan study examined principals’ interpretations and uses 
of external pressures and implementation of cooperative grouping. 

The general pattern to the findings is that teachers in schools led by Initiators 
and Managers have the most implementation success. Teachers in Responder 
CFS schools have much less implementation success. The findings also trend to- 
ward the most implementation success in schools with Initiator CFS principals. 

Implications of CFS for Facilitating Implementation 

The first implication from the work with CFS is that, just as with students 
and teachers, there are individual differences in principal leadership. All 
principals do nol facilitale implementation in the same way. If the findings 
from the CFS by teacher implementation success studies are accepted, then 
there are major implications for the provision of implementation support. 
Depending on the CFS of the principal, different types and intensities of 
facilitation will be needed. 
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An additional implication may be that careful consideration should be 
given to the selection of schools for alpha and beta field tests of new technol- 
ogies and applications. Responder CFS leaders will welcome the effort but 
provide little direct support for implementation. Manager CFS leaders will 
organize the resources and schedules to support implementation, assuming 
they have enough lead time. If Initiator CFS leaders like what the innovation 
purports to achieve, they will strongly support its implementation. However, 
if they do not agree with the promise of the innovation, then they will make 
sure it is not implemented in their school. 

Discussion, Impiications and Recommendations 
for Education Technoiogy Change 

The guest editor for this issue of JRTE, Neal Strudler, suggested that in this 
paper I “piece together best work and current overview of [my] research on 
change and highlight some key points and implications for effective integra- 
tion of technology. In other words, what educational technology advocates 
and researchers should know about their prospects for successful change” 
(Neal Strudler, e-mail communication, July, 2009). This is no easy task, espe- 
cially when considering that full books are written about the constructs and 
studies highlighted herein. 

As we know well, achieving change success is always a challenge, and 
even more so with technology. As Straub (2009) observed in a review of 
three adoption theories (Diffusion, Concerns-Based, and Technology Accep- 
tance), adoption of technology is “complex, inherently social, [and] develop- 
mental” (p. 625). The research-based constructs introduced above represent 
key approaches to understanding the complexity, addressing the human 
aspect, and documenting the developmental nature of change processes. 

Technology innovations add an additional complexity. Along with the 
usual issues and dynamics of change processes, technology innovations 
themselves keep evolving. Once developed and field tested, most innova- 
tions remain stable in terms of their ideal configuration. However, most 
technology innovations continue to evolve — in large and small ways. 
Sometimes it is a new device (e.g., a handheld or netbook), and other times 
it is an upgrade of a computer’s operating system or new software version. 
Each time, the delicate balance of the classroom lesson plan can be upset as 
a teacher tries to anticipate how to teach a math lesson on the new version of 
Excel, or instruct students on saving to a network server rather than a disc. 
Over time they add creative adaptations and make refinements, and the time 
between changes becomes shorter and shorter. A consequence for research- 
ers can be that the critical components of the alpha test configuration have 
been combined and supplemented in ways that emphasize different compo- 
nent by the time the epsilon version is ready for field testing. 

This paper has introduced one metaphor (the implementation bridge) 
and four research-based constructs that address elements of change. Each 
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of these provides conceptual and empirical foundations for drawing im- 
plications and recommendations to consider in achieving future successes 
with technology changes. Each also can contribute to explaining the limited 
extent of use and going-to-scale of past efforts. 

The Implementation Bridge 

The first assumption in our research has continued to be that change is a 
process, not an event. Many developers become such true believers of their 
innovation and the difference if can make fhaf fhey sfart assuming thaf ofh- 
ers will believe fhe same. The expecfafion is fhat everyone will aufomatically 
see fhe benefifs and adopf fhe new way, fhereby making change an event. An 
informative example of the consequences of maintaining this view was seen 
in the 1960s and 1970s wifh the national curriculum development projects. 
One of the stated themes was that each curriculum would be so up to date, 
so well evaluated, and so well packaged that it would be “teacher proof.” The 
expectation was that teachers would not make adaptations to these well- 
developed programs. Unfortunately, teacher proof had a different meaning 
in practice; most teachers didn’t use the new programs. 

Although the implementation bridge is a reminder that change is a pro- 
cess, it also provides an image of implementers needing to progress across 
the bridge. Three constructs and related measures have been introduced here 
that can be used to assess how far across the bridge each implementer has 
progressed. The same information can be used as diagnostics to determine 
what interventions will assist each implementer making further progress. 

An additional use of the metaphor is in thinking about implementation 
bridges having different lengths. Some implementation efforts are short, 
whereas others take years. Planning for change success should be different 
depending on the normative length of time required to achieve implementa- 
tion success. Updating a familiar piece of software should be quick, whereas 
learning to use a new operating system will take more time. Using one 
browser may be very much like another, but using a course management 
system or science simulation may prove more difficult. The ultimate goal of 
having integrated uses of technology by teachers and their students requires 
a longer bridge. 

Levels of Use 

Use of an innovation is not a dichotomous, yes/no phenomenon. The long- 
used, tried-and-true two-group research design does not make sense with 
human-use practices and innovations. The Levels of Use construct provides 
operational definitions of three distinguishable ways that an individual or 
group can be a nonuser and five ways an individual or group can be users. 
The traditional research design does not differentiate these different levels 
and simply views treatment (LoU 0, 1, and II) and comparison groups (LoU 
III-VI) as pure — a fatal flaw. 
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For example, one unacknowledged danger of the two-group study de- 
sign is the possibility that many/most/ all of the members of the treatment 
group will be first-time implementers. Change process studies consistently 
document that most first-time users of an innovation will be at LoU III: 
Mechanical Use. By definition, there is a disjointedness to their use, short- 
term thinking, and inefficiencies. There should be no surprise to finding no 
significant differences between groups when treatment/control studies are 
conducted with first-time implementers. Early implementers are engaged in 
trying to learn how to use the innovation. More surprising is that, in many 
of these studies, the comparison group doesn’t achieve more than the new 
way. Summative evaluations and treatment studies should be done with 
implementers that are beyond LoU III. 

There also are implications of LoU for facilitating change. For example, if 
a technology change has a number of parts, many routines, and complicated 
steps to its use, then it is easy to predict that implementers will spend more 
time at LoU III: Mechanical Use. The timeline for facilitating implementa- 
tion should anticipate this, and the plan should include resources, such as 
technology specialists assigned for support, to address and resolve the vari- 
ous mechanical-use problems that are likely to occur. The implementation 
game plan should also be in place for sufficient time so that most imple- 
menters can attain at least LoU IVA: Routine Use. Otherwise, sustainability 
is not likely to be achieved. 

Innovation Configurations 

It is likely that a range of configurations will be found in practice with most 
technology changes. In many instances, adaptation and innovative uses are 
encouraged. However, having more variation in configurations becomes 
a problem when there is a need to document results. Unless a particular 
configuration(s) associated with higher outcomes can be described, future 
implementers will not know which components and practices really are 
most critical to success. And innovations will receive credit for successes 
and failures that may not be deserved. For these reasons, developers should 
describe their ideal configurations clearly and in operational terms. Oth- 
erwise, it may have the same name and pieces, but the configurations that 
are implemented in different settings can range from high fidelity to drastic 
mutations of what the developer envisioned. 

This is particularly important as more schools and colleges of education 
invest in more technology (i.e., interactive whiteboards, handheld devices, 
laptop computers) that could be costly investments without clear under- 
standing of all implementation configurations. For example, in Donovans IC 
map of the laptop program, if you focused only on the “c” configuration, you 
would not want to expand the program. However, focusing on “a” and “b” 
configurations presents a more optimistic picture of how laptops can be used 
in the classroom. 
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Stages of Concern 

Failing to appreciate the importance of the personal side of change is prob- 
lematic. Many of the most promising innovations have been lost or have 
taken a very long time to go to scale because leaders have failed to attend 
to the feelings and perceptions of the implementers. Simply saying, “You 
should use this because it is good for students,” will not win over those 
teachers with Self or Task concerns. Stages of Concern provide a construct 
and measures for understanding the personal side of change from beginning 
to end of a change process. 

The dynamics of concerns are especially useful for those with responsi- 
bility to facilitate implementation. For example, quite often what is seen as 
resistance to change is related to having intense Stage 2; Personal concerns. 
Reduction of high personal concerns requires being empathetic and pro- 
viding information. However, a change process will go much more easily if 
personal concerns do not get high in the first place. This means anticipating 
the potential for the arousal of personal concerns and, early on in the change 
process, providing relevant information that addresses Stage 1: Informa- 
tional concerns. 

Change Facilitator Style 

The fourth research-based construct introduced in this article. Change Fa- 
cilitator Style, addresses the importance of administrator leadership. All too 
often, innovation advocates think only about the front-line implementers, 
teachers in classrooms. Administrators are addressed to obtain the adop- 
tion decision, then nearly all attention is devoted to installation, training, 
and support for the implementers. An important reality is that teachers are 
members of organizations, and leadership is a significant factor in their 
having change success. The research on principal CFS offers a framework for 
distinguishing different approaches to change leadership. The different CFS 
are significantly correlated with teacher implementation success (0.76 in the 
original study). Implementers with Responder CFS leaders have less change 
success. In most studies, implementers in schools led by Initiators have the 
most success. 

Keep in mind that three CF Styles have other implications. For example. 
Responder leaders are most willing for new efforts to be introduced in their 
schools. They say, “Go ahead. We like trying new ideas in this school.” How- 
ever, their follow-through and support will be limited. Managers will ensure 
that there are sufficient resources and all rules and procedures are followed. 
They say, “My teachers are very busy. If you give me a grant or another posi- 
tion, then we can do it.” Be sure to have clear budget proposals and detailed 
timelines. Initiators will check to see how well the proposed innovation aligns 
with his/her vision and directions for the school. They ask, “How will this fit 
with our priorities to increase students’ higher-order learning?” Information 
about outcomes will be important with Initiators. With each style of principal, 
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gaining entry requires addressing different questions, and achieving imple- 
mentation success requires thinking beyond initial ease of access and provid- 
ing different levels and types of implementation facilitation. 

Consideration of the Uitimate Question: Increasing Outcomes 

As important as “selling them” and “getting them to use” technology may 
be, the end goal is still to see improvements in outcomes. With changes in 
educational technology, the bottom line has to be determining what happens 
with student outcomes. An important purpose of this paper is to recom- 
mend careful attention to key implementation factors as part of examining 
outcomes. After all, unless the users have worked through the mechanical 
problems of use, implemented fidelity configurations, and reduced Self and 
Task concerns, any assessment of student effects will be misleading. 

As a change process researcher, I regularly recommend a paradigm shift 
when it comes to selecting a research design. Pure treatment and control- 
group designs rarely, if ever, make sense in schools, or in other organization- 
al settings, for that matter. At its most absurd, the “gold standard” research 
paradigm demands that one group of kids be taught and another group not 
be taught to read. In reality, all comparison groups will have some sort of 
practices in place that address the targeted student outcomes. 

A more logical approach to research and evaluation studies is to compare 
higher to lower levels of implementation. In practice, this entails assess- 
ing how far across the bridge each implementer has moved and comparing 
outcomes each is obtaining. This is the approach used in the mathematics 
study referenced above (George, Hall & Uchiyama, 2000). In that study, 
higher-fidelity configurations and higher Levels of Use were associated with 
significantly higher student test scores. 

If some sort of dichotomy needs to be included (because that is what the 
grant office expects), then compare those who have moved all the way across 
the bridge (LoU IVA and above) with those who have not stepped on the 
bridge (LoU 0, 1, and II). If a good-quality IC map has been constructed, the 
right-hand component variations — the “d’s,” “es,” and “f s” — will represent 
traditional practices. (This is one of the ways that an IC map differs from 
typical rubrics.) Comparisons of outcomes can then be made between “a-b” 
configurations and “d-e” configurations. Either way, the implementation 
bridge offers a research design that is based in the realities of implementation. 

In summary, at this time education technology scholars and practitioners 
are engaged with some of the most promising and interesting innovations. 
They also have been confronted first hand with the challenges associated 
with disappointing implementation efforts and failures to go to scale. In 
most instances, it appears to this author that the main causes of failure have 
not been the technology innovations, but rather that the failures have had 
more do to with underappreciating the challenges of implementation. The 
needs are so high for schools to improve student outcomes and the promises 
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SO powerful with integrated uses of technology that the challenges of imple- 
mentation must be overcome. Besides, it will be very disappointing if today’s 
efforts to have widespread integrated uses of technology fail and become just 
another chapter in Larry Cuban’s next book. 
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